
IMAGE PROCESSING APPARATUS 
BACKGROUND OF THE INVENTION 

1. Field of the Invention 

The present invention relates to an image processing apparatus for 
detecting three-dimensional position and posture (orientation) of an object, 
and in particular to an image processing apparatus suitable for use in a bin- 
picking operation for taking out a workpiece one by one from a stack of 
workpieces using an industrial machine such as a robot. 

2. Description of Related Art 
operation of taking out an individual workpiece form a randomly 

arranged s^ack of workpieces or an aggregation of workpieces contained in a 
container orb predetermined size, which have identical shapes and different 
three-dimensional positions/postures, have been performed manually. In 
storing workpietes in a pallet or placing workpieces at a predetermined 
position in a machine or a device using a (dedicated) robot, since it has been 
impossible to direcrfr take out an individual workpiece one by one from the 
randomly arranged stack of workpieces by the dedicated robot, it has been 
necessary to rearrangeVhe workpieces in advance so as to be picked out by the 
robot. In this rearrangement operation, it has been necessary to take out an 
individual workpiece from the stack manually. 

The reason why individual workpieces having identical shapes and 
different three-dimensional positions/postures can not be picked out by a robot 
from a randomly arranged stack of workpieces or an aggregation of 
workpieces contained in a container is that the position/posture of individual 
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workpieces in the stack or the aggregation can not be recognized, so that a 
robot hand can not be placed to a suitable position/posture at which the robot 
hand can hold the individual workpiece. 



SUMMARY OF THE INVENTION 

An object of the present invention is to provide an image processing 
apparatus capable of detecting three-dimensional position and posture of 
individual objects in a randomly arranged stack or an aggregation in a 
container of a predetermined region, which have identical shapes and different 
Jhree-dimensional positions/postures. 

X ^n image processing apparatus of the present invention comprises 
rapturing device; and a memory storing reference models created 
based on image data of a reference object captured by the image capturing 
device in a plurality of directions, and storing information of the capturing 
directions to bet respectively associated with the reference models. The 
reference objectVnay be an object of detection itself or an object having a 
shape identical to that of the object of detection. 

The image pk>cessing apparatus also comprises a processor to perform 
matching processing oX image data containing an image of the object of 
detection captured by theSimage capturing device with the reference models to 
select an image of an objecV matched with one of the reference models, and to 
obtain posture, or posture anovposition of the object based on the selected 
image of the object, said one reference model and the information of the 
direction associated with said oneVeference model. 

The reference models may be a part of the image data of the reference 



object or obtained by processing the image data of the reference object. 
S 

^)^^p y^^Hf image capturing device may be a camera for capturing two- 

lage data, and in this case the image data of the reference object 
are captured tty the image capturing device from a predetermined distance. 
Alternatively, toe image capturing device may be a visual sensor for capturing 
three-dimensional^mage data, and when the three-dimensional visual sensor is 
adopted the image data containing an image of the object of detection may be 
two-dimensional arrangement data including distance information from the 
object of detection to thAimage capturing device, a part of said two- 
dimensional arrangement olata or a set of distance data. 

The image capturing device may be attached to an wrist of an robot. 
Further, the image data of the reference object can be captured in a place 
different from a place where the detection of the object is performed, and 
supplied to the image processing apparatus on line or offline. 

BRIEF DESCRIPTION OF THE DRAWINGS 



FIG. 1 is a diagram for showing a picking operation by a robot to take 
out an individual workpiece from a stack of workpieces using an image 
processing apparatus according to an embodiment of the present invention; 
^ , ^^FI GS^ 2a-2d show an example of iefeieiice T553etsr 

FIG. 3 is a block diagram of a principal part of a robot controller; 

FIG. 4 is a block diagram of the image processing apparatus according 
to an embodiment of the present invention; 

FIG. 5 is a flowchart of the processing for creating reference models; 

FIG. 6 is a flowchart of the processing for the picking operation; 
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FIG. 7 is a diagram showing an example of scanning motion of a 
visual sensor capable of obtaining distance data; 

FIG. 8 is a diagram of the two-dimensional arrangement data 
containing distance data as image data obtained by the visual sensor; 

FIG. 9 is a flowchart of processing for obtaining the two-dimensional 
arrangement data. 



DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 



ifivention is used in combination with an robot system will be described. In 
this embodiment, an image of a stack of workpieces, which are objects of 
detection havW identical shapes and randomly arranged as shown in FIG. 1, 
is captured by an image capturing device (camera or visual sensor) 20, which 
is attached to a wrist of a robot RB and position and posture (orientation) of 
the individual workpieces are detected based on the captured image. For this 
purpose, images of a reference object, which is one of workpieces W subjected 
to a picking operation or an object having a shape identical to that of the 
workpiece W are captured in different directions by the image capturing 
device and reference models are created from the image data obtained by the 
image capturing and storeti in advance. Matching processing between the 
image data obtained by capturing the image of the stack of workpieces and the 
reference models is executemto select an image of one workpiece matched 
with one of reference models,Wd a position/posture of the selected workpiece 
is determined based on the selected image of the workpiece in the image field 
of view, the selected one of taugmt modes and the position/posture information 




embodiment in which an image processing apparatus of the present 
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associated with selected one of the reference models. 

FIG. 3 is a block diagram showing a principal part of a robot 
controller 10 for use in the embodiment of the present invention. A main 
processor 1, a memory 2 including a RAM, a ROM and a nonvolatile memory 
(such as an EEPROM), an interface 3 for a teaching operating panel, an 
interface 6 for external devices, an interface 7 for an image processing 
apparatus and a servo control section 5 are connected to a bus 8. A teaching 
operating panel 4 is connected to the interface 3 for a teaching operating panel. 

A system program for supporting basic functions of the robot RB and 
robot controller 10 are stored in the ROM of the memory 2. Robot operation 
programs and their related determined data which are taught in accordance 
with various operations are stored in the nonvolatile memory of the memory 2. 
The RAM of the memory 2 is used for temporarily storage of data for various 
arithmetic operations performed by the processor 1. 

/^^ e servo control section 5 comprises servo controllers 5al to San (n: 
Sum of th&number of all the axes of the robot including additional movable 
axes of a tool attached to a wrist of the robot), each composed of a processor, a 
ROM, a RAMi etc. Each servo controller performs position/velocity loop 
control and also Wrent loop control for its associated servomotor for driving 
the axis, to functioii as a so-called digital servo controller for perfonninng 
loop control of position, velocity and current by software. Each servomotor 
Ml-Mn for driving each axis is drivingly controlled according to outputs of 
the associated servo controller 5al-5an through the associated servo amplifier 
5bl-5bn. Though not shown in FIG. 3, a position/velocity detector is attached 
to each servomotor Ml -Mm and the position and velocity of each servomotor 
detected by the associated position/velocity detector is fed back to the 
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associated servo controller 5a^ -5an. To the input/output interface 6 connected 
are sensors of the robot, and actuators and sensors of peripheral devices. 
FIG 4 is a block diagra n of the image processing apparatus 30 

le robot controller 10. The image processing 
apparatus 30 comprises a processor 31 to which a ROM 32 for storing a 
system program to be executed by the processor 31, an image processor 33, an 
image-capturing- device interface 34 connected to the image capturing device 
20, a MDI 35 with a display sudh as a CRT or a liquid crystal display for 
inputting and outputting various Icommands and data, a frame memory 36, a 
nonvolatile memory 37, a RAM B8 for temporary storage of data and a 
communication interface 39 for tHp robot controller are connected. An image 
captured by the camera 20 is storeu in the frame memory 36. The image 
processor 33 performs image processing of images stored in the frame memory 
36 on demand of the processor 31 s© as to recognize an object. The 
architecture and function of the image processing apparatus 30 itself is no way 
different form the conventional imagevprocessing apparatus. The image 
processing apparatus 30 of the presenfl invention is different form the 
conventional one in that reference models as described later are stored in the 
nonvolatile memory 37 and pattern matching processing is performed on an 
image of a stack of workpieces W captured by the image capturing device 20 
using the reference models to obtain a position and posture of a workpiece W. 

The image capturing device 20 is used for obtaining image data, as 
described later, and may be a CCD came ra for obtaining two-dimensional 
images data or a visual sensor capable on obtaining three-dimensional image 
data including distance data. In the case of using the CCD camera, the image 
data is obtained by a conventional method based on two-dimensional images 




captured by the CCD camera, but in case of the visual sensor capable of 
obtaining three-dimensional data including distance data, two-dimensional 
arrangement data with distance d ata between the sensor and an object is 
obtained. The visual sensor for obtaining the three-dimensional data including 
distance data is known, for exan pie, from a three-dimensional visual sensors 
of a spot light scanning type disclosed in Japanese Patent Publication No. 
7-270137, and the summary of tpe three-dimensional visual sensor is 
described below. 

This visual sensor detedts a three-dimensional position of an object by 
irradiating a light beam to foni a light spot on the object for scanning the 
object-in two different directions (X direction and Y direction) and by 



detecting the light reflected on 



using the respective inclination 



the object by a position sensitive detector 



(PSD). Three dimensional pos ition of the object is measured by a calculation 



angles 0x, 0y of mirrors for scanning and an 



incident positions of the reflected light beam on the PSD. 

Referring to FIGS. 7-9, a method of obtaining two-dimensional 
arrangement data including distance data using the three-dimensional visual 
sensor will be explained briefly. 

anning range (measuring range) on an object is set in advance, and 
inclination angle 0x, 0y of the mirrors is controlled discretely. As shown in 
FIG. 7, the scanning is performed from a point (1,1) to a point (1, n), from a 
point (2,1) to a point (2, n), from a point (m, 1) to a point (m, n) on the X-Y 
plane within the scanning range, to measure three-dimensional positions of 
each reflected poinrton the object. Also, a distance Z (i, j) between the sensor 
and the reflection point (i, j) on the object is obtained and stored in the 
RAM38 of the image processing apparatus 30. Thus, the image data is 




obtained as two-dimens onal arrangement data including the distance data Z (i, 
j) between the sensor ai d the reflection point on the object, as shown in FIG. 

8. . 

FIG. 9 is a flowchart of processing to be executed by the processor 3 1 
of the image processing apparatus 30 for obtaining the image data. 

First, indexes i and j are respectively set to "1" (Step 300) and the 
inclination angle (0x, 0y) of the mirrors is set to (xl, yl) to direct to the start 
point (1,1) and an irradiation command with the inclination angle is send to 
the sensor 20 (Steps 301-303). The sensor irradiates a light beam with the 
mirrors set at the inclination angle. The signal representing the image 
captured by the PSD is sent to the image processing apparatus 30. The 
processor 3 1 of the image processing apparatus 30 calculates the position of 
the reflection point on the object from the signal from the PSD and the 
inclination angle (0x, 0y) of the mirrors to obtain the distance Z (i, j) between 
the sensor and the position of the reflection point on the object. This value Z 
(i, j) is stored in the RAM 38 as the two-dimensional arrangement data [i, j] 
(Step 304, 305). The calculation for obtaining the position of the reflection 
point and the distance Z (i, j) may be performed by the sensor 20. 

-OThen, the index i is incrementally increased by "1" and the inclination 

w/ \ 

Wangle 0x of the mirror for X-axis direction scanning is increased by the 
predetermined amount Ax (Step 306, 307). It is determined whether or not the 
index i exceeds the set value n (Step 308). If the index i does not exceed the 
set value n, theVrocedure returns to Step 303 and the processing from Step 
303 to Step 308 lk executed to obtain the distance Z (i, j) of the next point. 
Subsequently, the processing of Steps 303-308 are repeatedly executed until 
the index i exceeds tnfe set value n to obtain and store the distance Z (i, j) of 
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the respective points (Jl, 1) to (1, n) shown in FIG. 7. 

If it is determined that the index i exceeds the set value n in Step 308, 
the index i is set to "11 and the index j is incrementally increased by C T ? to 
increase the inclinationWgle By of the mirror for Y-axis direction scanning 
(Steps 309-31 1). Then,\t is determined whether or not the index j exceeds the 
set value m (Step 312) arm if the index j does not exceed the set value m, the 
procedure returns to Step A02 to repeatedly executes the precessing of Step 
302 and the subsequent Stem. 

Thus, the processing nom Step 302 to Step 312 is repeatedly executed 
until the index j exceeds the setValue m. If the index j exceeds the set value 
m, the points in the measurementVange (scanning range) shown in FIG. 7 has 
been measured entirely, the distancfe data Z (1, 1) - Z (m, n) as two- 
dimensional arrangement data are stored in the RAM28 and the image data 
obtaining processing is terminated. A part of the image data of two- 
dimensional arrangements or a plurahty \f distance data can be obtained by 
appropriately omitting the measurement orthe distance for the index i. 

The foregoing is a description on the processing for obtaining two- 
dimensional arrangement data as image data using the visual sensor capable of 
measuring the distance. Using the two-dimensional arrangement data obtained 
in this way as image data, creation of reference models and detection of 
position and posture (orientation) of an object can be performed. In order to 
simplify the explanation, the following description will be made assuming that 
a CCD camera 20 is used as an image capturing device and the two- 
dimensional image data obtained by capturing image of the object by this 
camera 20 is used. 

Processing for creating reference models will be explained referring to 
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FIGS. 2a-2d and FIG. 5. FIG. 5 is a flowchart showing processing for 
teaching reference models to the image processing apparatus 30 according to 
the present invention. 

One reference workpiece (one of the worpieces W as objects of robot 
operation or a workpiece having a three-dimensional shape identical to that of 
the workpiece W) is prepared for creating reference models. A first (0-th) 
position/posture of the reference workpiece at which the camera 20 attached to 
a distal end of a robot wrist captures the image of the object is set, and an axis 
of rotation and rotation angles with respect to the first (0-th) position/posture 
are set in order to determine the subsequent positions/postures of the reference 
workpiece. In addition, the number of the positions/postures of the workpiece 
at which the camera 20 captures the image of the object are set. In this 
example, information of both position and posture is used, however it is 
sufficient for creating reference models using only posture (orientation) 
information if a demanded precision of position is not high. 

As shown in FIGS. 2a to 2d, in this example, images of the reference 
workpiece are captured from four different directions and reference models are 
created based on the four image data. As shown in FIG. 2a, an image of the 
reference workpiece is captured from the direction of a Z-axis of a world 
coordinate system at 0-th position/posture to create 0-th reference model. 
For setting the subsequent positions/postures, an axis perpendicular to an 
optical axis of the camera and passing a central point of the workpiece (an 
origin of a work coordinate system set to the workpiece) and rotation angles of 
the workpiece along the rotation axis are set for this camera position. Since 
the optical axis of the camera is set parallel to the Z axis of the world 
coordinate system, an axis parallel to either the X-axis or the Y-axis of the 
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world coordinate system, which is perpendicular to the Z axis, can be selected 
and the workpiece is rotated around the rotation axis at the workpiece position. 

In the example, an axis parallel to the X-axis of the world coordinate 
system is set as the rotation axis, and for the position/posture shown in FIG. 
2b, the rotation angle of 30° is set to rotate the workpiece by 30° with respect 
to the camera along the rotation axis. 1st reference model is created based on 
the image data of the workpiece at the position/posture shown in FIG. 2b. 
Similarly, as shown in FIGS. 2c and 2d, the workpiece is rotated by 60° and 
90°, respectively, along the rotation axis for capturing images of the 
workpiece to create 2nd and 3rd reference models. 

In this example, rotation angles of zero degree, 30 degrees, 60 degrees 
and 90 degrees are set for creating four reference models. The dividing range 
of the rotation angles may be set more finely and/or range of the rotation angle 
may be set greater to create more reference models for more precise detection 
of the position/posture of the workpiece. 



Camera 20 captures the image of the object, and the rotation axis and the 
rotation anglesswith respect to the 0-th position/posture are set in advance in 
order to detenninfe the subsequent positions/postures of the reference 
workpiece, and alsoSdie number of the subsequent positions/postures of the 
workpiece are set. Fonuitelligible explanation, it is assumed that an optical 
axis of the camera is parallel to the Y-axis of the world coordinate system and 
that a position where the X-a&s and Y-axis coordinate values are identical to 
those of the reference workpieck and only the Z-axis coordinate value is 




Processing of creating the four reference models will be explained 
referring to flowchart of FIG. 5. 

/*&s described above, the 0-th position/posture of the robot at which the 



teaching operation 



different form that of] the position of the reference workpiece is taught to the 
robot as the 0-th image capturing position for obtaining the 0-th reference 
model. Further, the positions of the robot where the camera is rotated with 
respect to the refere ace workpiece by 30 degrees, 60 degrees and 90 degrees 
along the axis passi ag the central point of the reference workpiece and parallel 
to the X-axis of the world coordinate system are set as the 1st, 2nd and 3rd 
image capturing po sition, and the number N of the image capturing positions 
is set "4." 

When a coifimand of creating reference models is inputted from 

>anel 4, the processor 1 of the robot controller 10 sets a 
counter M for countnng the number of the image capturing to "0" (Step 100). 
The robot is operateu to have the M-th position/posture and a command for 
image capturing is outputted to the image processing apparatus 30 (Step 101). 
In response to this command, the image processing apparatus 30 performs 
capturing of an imaga of the reference workpiece with the camera 20 and the 
captured image data it stored in the frame memory 36. Further, relative 
position/posture of tha workpiece with respect to the camera is obtained and 
stored in the nonvolatile memory 37 as relative position/posture of M-th 
reference model, and aldata-captured signal is sent to a robot controller (Step 
103). Thus, position/pasture of the workpiece in a camera coordinate system 
set to the camera is obt|ained from the position/posture of the camera and the 
position/posture of the Reference workpiece in the world coordinate system 

;e by the camera, and is stored as the relative 
position/posture of the ^orkpiece with respect to the camera. For example, the 
position/posture of the workpiece in the camera coordinate system is stored as 
[xO, yO, zO, a0, p0, y0]Q, where a, p and y mean rotation angle around X-, Y-, 



when capturing the im; 



Z- axes, and "c" means the camera coordinate system. 

Upon receiptW the data-captured signal, the processor 1 of the robot 
controller 1G incremeitally increase the value of the counter M by "1 "(Step 
104) and determine whether or not the value of the counter M is less than a set 
value N (=4) (Step lOsV If the value of the counter Mis less than the set 
value N, the procedure leturns Step 101 to move the robot to the M-th image- 
capturing pdsition/postule. Thus, in the example as shown in FIGS. 2a-2d, the 
camera is successively tulned by 30 degrees around the axis parallel to X axis 
of the world coordinate sistem and passing the workpiece position, and 
successively captures the Wage of the workpiece, and reference models and 
relativity positions/posture^ of the camera with respect to the workpiece at the 
image capturing are stored^ 

Processing of Steps joi-105 are repeatedly executed until the value of 
the counter M equals to the set value N (=4), and the reference models and the 
relative positions/postures o| the camera and the workpiece are stored in the 
nonvolatile memory 37. This, the reference models created from the image 
data of the workpiece at the positions/postures shown in FIGS. 2a-2d are 
stored, and the relative posil ions/postures between the camera and the 
workpiece for respective reference models are stored as positions/postures of 
the workpiece W in the camdra 



a coordinate system as [x 0, y 0, z 0, a0, po, 
l]c, [x 2, y 2, z 2, a2, 02, Y 2]c, and [x 3, y 3, z 3, 



Y0]c,[xl iy i,zl,al,pl,-y 
«3, P3, Y 3]c. 

The reference models 'and the relative position/posture of the 
workpiece W and the camera 20 are stored in the nonvolatile memory 37 of the 
image processing apparatus 30. In the above described embodiment, the 
reference models are created using a robot, however, the reference models may 
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be created by a manual operation without using a robot. In this case, the 
reference workpiece is arranged within a field of view of the camera connected 
to the image processing apparatus 30, and the images of the workpiece with 
different postures are captured by the camera. The reference models are 
created based on the image data and the relative positions/postures of the 
camera and the workpiece at the image capturing manually inputted, and are 
stored with the respective relative positions/postures. 

The reference models may be created from a part of the image data of 
the reference object, and may be created by processing the image data of the 
reference object. 

In addition, the reference models may be created based on the stored 
image data of the reference workpiece when detecting the position/posture of 
the objective workpiece, without creating and storing the reference models in 
advance. 

Hereinafter, a picking operation for taking out an individual workpiece 
by a robot from a stack of workpieces each having a shape identical to that of 
the reference workpiece will be described, as an example of a method of 
detecting three-dimensional position/posture of an object, using the image 
processing apparatus 30 storing the reference models. 

Kj. 6 is a flowchart of processing for the picking operation. When a 
Sking command is inputted into the robot controller 10 from the teaching 
operation paneL4, the processor 1 operates the robot RB to move the camera 
attached to the roltat wrist to an image capturing position where a stack of 
workpieces are within a field of view of the camera 20 (Step 200). Three- 
dimensional position/pbsture of the camera 20 on the world coordinate system 
at this image capturing position is outputted to the image processing apparatus 
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30, and a image capturing command is outputted (Step 201). Upon receipt of 
the image capturing command, the processor 3 1 of the image processing 
apparatus 30 captures an image of the stack of the workpieces W, to obtain 
image data of some workpiejbes W and store it in the frame memory 36 (Step 
202). . 

Then, pattern matching processing is performed for the image data 
stored in the frame memory! 36 using one of reference models (1st reference 
model) stored in the nonvolatile memory 37 so as to detect a workpiece W 
(Step 203). In this pattern matching processing, matching of the image data of 
the reference model with the image data of workpieces is performed on the 
basis of position, turn and scale. It is determined whether or not an object 
having a matching value eqi al or greater than the set value (Step 204). If an 
object having a matching va ue equal or greater than the set value is not 
detected, the procedure proc ^eds to Step 205 to determine whether or not the 
pattern matching is performed using all the reference models (1st to 4th 
reference models). If the pattern matching using all the reference models is 
not yet performed, further pittern matching is performed using another 
reference model (Step 206) 



If it is determined ii 
equal or greater than the se 



Step 204 that an object having a matching value 
value with respect to any of the reference models 
is detected, the procedure proceeds to Step 207 to perform matching 
processing on the two-dimeikional data of the detected workpieces W, using 
every taught mode. In Step 208, the reference model having the most large 
matching value in the pattern matching processing is selected, and the relative 
position/posture of the workp iece W with respect to the camera 20 is 
detennined based on the relative position/posture of the camera and the 
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processing, (Step 208). The po 



/ 

reference workpiece stored for t le selected reference model, and position, 
rotation angle and scale of the ii aage of the workpiece in the matching 

lition and posture (orientation) of the detected 
workpiece on the world coordinate system is determined from the position and 
posture of the camera 20 in the/ world coordinate system, which has been sent 
in Step 201, and the relative plsition/posture of the workpiece W with respect 
to the camera 20, and is outpuped (Step 209). Thus, since the relative 
position/posture of the workpiece W with respect to the camera 20 is the 
position/posture of the workp: ece W in the camera coordinate system, the 
position and posture (orientation) of the detected workpiece W in the world 
coordinate system is obtained 1 by an arithmetic operation of coordinate 
transformation using the data of the position/posture of the workpiece W in the 



camera coordinate system an< 



the position/posture of the camera 20 in the 



world coordinate system (Stew 209). 



The reference model having the highest matching value is selected in 
this embodiment, however, a reference model of the rotation angle of zero 
degree (the 0-th reference model) may be selected with precedence, or an 
object having the highest expansion rate of scale (the object which is nearest 
to the camera, i.e. located at the summit of the stack in this example) may be 
selected with precedence. 

The robot controller 10 operates the robot to perform a picking 
operation to grip and hold the detected workpiece W and move the held 
workpiece W to a predetennined position, based on the three-dimensional 
position/posture of the workpiece W (Step 210). Then, the procedure returns 
to Step 202 to repeatedly execute the processing of Step 202 and subsequent 
Steps. 
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When all the workpieces have been picked form the stack of the 
workpieces, matching value equal to or greater than the set reference value 
cannot be obtained in the pattern matching processing for all reference models 
in Steps 203-206, and the picking operation is terminated. 

In the case where a stack of the workpieces can not fall within the field 
of view of the camera 20, or in the case where it is not necessary to capture an 
image of a workpiece behind other workpieces by changing the orientation of 
the camera, the procedure may return to Step 200 when it is determined "Yes" 
in Step 205, to move the camera to another position/posture at which an image 
of the objective workpiece can be captured. 

In addition, in the case where the robot and the image processing 
apparatus 30 are used in combination as in the foregoing embodiment, 
the robot controller 10 may store the three-dimensional position/posture of the 
camera without outputting it to the image processing apparatus 30 in Step 201 
and the relative position/posture of the workpiece and the camera may be 
outputted from the image processing apparatus 30 to the robot controller 10 in 
__Step208 to execute the processing of Step 209 in the robot controller 10. 
^^y/^rther, in the case where a wide-angle lens is installed in the CCD 
/camera as toe image capturing device, for example, there is possibility of 
judging the inWation angle to be 30 degrees by influence of parallax when a 
workpiece of zerWegree inclination is at a corner of a field of view of the 
camera. In such a dase, the camera may be moved parallelly in accordance 
with the position of thWorkpiece in the field of view of the camera to a . 
position right above the Vrkpiece to lose influence of parallax, and at this 
position the image capturinWocessing of Step 201 and the subsequent Steps 
in FIG. 6 is performed so thatNhe false judgment is prevented. 




Furthermore, in order to obtain three-dimensional position/posture of 
an objective workpiece whose three-dimensional position/posture is unknown 
without using a robot, the camera is arranged to capture an image of a stack of 
workpieces or a region containing the objective workpiece within a field of 
view of the camera, and the position/posture of the camera in the world 
coordinate system is inputted to the image processing apparatus 30 and issuing 
an object detection command to the image processing apparatus 30, to make 
the image processing apparatus 30 execute Steps 202-209 of FIG. 6. 

The image data for creating the reference models may be obtained at a 
place different form the place where the robot is installed. In this case, the 
image data may be supplied to the image processing apparatus on line through 
a communication interface provided in the image processing apparatus, or may 
be supplied offline through a disc driver for reading a floppy disk, etc. 

According to the present invention, a position/posture of an objective 
workpiece in a randomly arranged stack of workpieces or an aggregation of 
workpieces gathered in a predetermined region which have identical shapes 
and different three-dimensional positions/postures is detected, to thereby 
enable a robot to automatically pick out an individual workpiece from such a 
stack or an aggregation. 
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